Skip to content

chore: Remove support for 3.5.7 and 4.0.1#695

Open
Maleware wants to merge 3 commits into
mainfrom
chore/spark-update-version-26.7.0
Open

chore: Remove support for 3.5.7 and 4.0.1#695
Maleware wants to merge 3 commits into
mainfrom
chore/spark-update-version-26.7.0

Conversation

@Maleware

@Maleware Maleware commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Description

Removing spark versions 3.5.7 and 4.0.1, deprecate 3.5.8 and LTS 4.1.1.

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

  • Links to generated (nightly) docs added
  • Release note snippet added

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Feature Tracker has been updated
  • Proper release label has been added
  • Links to generated (nightly) docs added
  • Release note snippet added
  • Add type/deprecation label & add to the deprecation schedule
  • Add type/experimental label & add to the experimental features tracker

@Maleware

Maleware commented Jun 10, 2026

Copy link
Copy Markdown
Member Author

Solves: stackabletech/docker-images#1502

@Maleware Maleware mentioned this pull request Jun 10, 2026
Open
13 tasks
@Maleware Maleware changed the title Removing and , deprecate , LTS chore/bump-spark-versions Jun 10, 2026
@Maleware Maleware self-assigned this Jun 11, 2026
@Maleware

Maleware commented Jun 11, 2026

Copy link
Copy Markdown
Member Author

Release Snipped:

New LTS version: 4.1.1

Deprecated Versions: 3.5.8
Removed Versions: 3.5.7, 4.0.1

@Maleware Maleware marked this pull request as ready for review June 11, 2026 07:54
@Maleware Maleware moved this to Development: Waiting for Review in Stackable Engineering Jun 11, 2026
sbernauer
sbernauer reviewed Jun 11, 2026
View reviewed changes

@sbernauer sbernauer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!
Can you pls:

  • Adjust the PR title (e.g. chore: Remove support for 3.5.7 and 4.0.1)
  • Make sure the getting started guide still works (as we did a major version bump)

Comment thread docs/modules/spark-k8s/partials/supported-versions.adoc

- Missing HBase compatibility (See: https://github.com/apache/hbase-connectors/pull/130)
- No Iceberg Spark runtime release with support for Spark 4.1 available yet.
- No Delta Lake release with support for Spark 4.1 available yet.

@sbernauer sbernauer Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you pls check if that is still the case?

@Maleware Maleware Jun 11, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I've overlooked those. It seems to be not the case anymore: https://delta.io/blog/2026-03-01-delta-lake-4-1-0-released/ Thank you!

Comment thread docs/modules/spark-k8s/partials/supported-versions.adoc
Apache Spark 4.1.1 has the following known issues (as of February 2026):

- Missing HBase compatibility (See: https://github.com/apache/hbase-connectors/pull/130)
- No Iceberg Spark runtime release with support for Spark 4.1 available yet.

@sbernauer sbernauer Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.11 now supports it

Suggested change

Comment thread docs/modules/spark-k8s/partials/supported-versions.adoc

Apache Spark 4.1.1 has the following known issues (as of February 2026):

- Missing HBase compatibility (See: https://github.com/apache/hbase-connectors/pull/130)

@sbernauer sbernauer Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: This is now a 404 :( Maybe you can do some digging what happened there

@razvan razvan Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know what happened there but Spark4 is still not supported apache/hbase-connectors#155

Comment thread docs/modules/spark-k8s/partials/supported-versions.adoc
- 4.1.1 (Hadoop 3.4.2, Scala 2.13, Python 3.12, Java 21) (LTS)
- 3.5.8 (Hadoop 3.4.2, Scala 2.12, Python 3.11, Java 17) (Deprecated)

Apache Spark 4.1.1 has the following known issues (as of February 2026):

@sbernauer sbernauer Jun 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls update to Juli 2026 once you checked again :)

@sbernauer sbernauer moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Jun 11, 2026
@Maleware Maleware changed the title chore/bump-spark-versions chore: Remove support for 3.5.7 and 4.0.1 Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@razvan razvan razvan left review comments

@sbernauer sbernauer sbernauer left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@Maleware Maleware

Labels

release/26.7.0

Projects

Stackable Engineering
Status: Development: In Review

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Maleware @razvan @sbernauer